In cloud data centers, there is often the challenge of finding a good balance between the costs of operation and offering services to customers in a way that the customers expect. Therefore, it is usual to setup some services in a data center as a shared service. Typical use case are services for asynchronous processing such as an event server or job servers. Such servers may use queues that act on the FIFO (first in first out) principle. For some use cases, this behavior may be sufficient. However, in a data center environment situations often arise that are more demanding.
Consider the following situation. Suppose a data center has 100 customers. Of those customers, suppose in a given instance that 70 customers are using the data center with average loading on the data center, 25 customers are not using the data center much, and 5 customers happen to be preloading the system with data, for example, because they will be coming on-line soon. The preload activities typically occur asynchronously and so they can fill up the queue of the event server. If these asynchronous tasks have also long processing times, this can lead to a situation where the simpler asynchronous tasks of the other customers may be unacceptably delayed before being processed.